Inter-Rater Agreement (IRA)

Authors

Affiliations

Nate Yomogida, B.S., SPT

Doctor of Physical Therapy

B.S. in Kinesiology

Chloë Kerstein, B.S., SPT

Doctor of Physical Therapy

B.A. in Neuroscience

Inter-rater Agreement (IRA)

• Agreement is defined as the degree to which scores/ratings are identical
• IRA indices, therefore, relate to the extent to which different raters assign the same precise value for each¹

Tests of inter-rater agreement

• Strictly IRA measures: Bland-Altman plots¹
• others have been used in the literature as measures of both IRA and IRR (eg, Cohen’s kappa)¹

References

1.

Gisev N, Bell JS, Chen TF. Interrater agreement and interrater reliability: Key concepts, approaches, and applications. Research in social & administrative pharmacy: RSAP. 2013;9(3):330-338. doi:10.1016/j.sapharm.2012.04.004